{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "分类应用的目标是，根据已知类别的数据集，经过训练得到一个分类模型，再用模型对类别\n",
    "未知的数据进行分类。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iris Plants Database\n",
      "====================\n",
      "\n",
      "Notes\n",
      "-----\n",
      "Data Set Characteristics:\n",
      "    :Number of Instances: 150 (50 in each of three classes)\n",
      "    :Number of Attributes: 4 numeric, predictive attributes and the class\n",
      "    :Attribute Information:\n",
      "        - sepal length in cm\n",
      "        - sepal width in cm\n",
      "        - petal length in cm\n",
      "        - petal width in cm\n",
      "        - class:\n",
      "                - Iris-Setosa\n",
      "                - Iris-Versicolour\n",
      "                - Iris-Virginica\n",
      "    :Summary Statistics:\n",
      "\n",
      "    ============== ==== ==== ======= ===== ====================\n",
      "                    Min  Max   Mean    SD   Class Correlation\n",
      "    ============== ==== ==== ======= ===== ====================\n",
      "    sepal length:   4.3  7.9   5.84   0.83    0.7826\n",
      "    sepal width:    2.0  4.4   3.05   0.43   -0.4194\n",
      "    petal length:   1.0  6.9   3.76   1.76    0.9490  (high!)\n",
      "    petal width:    0.1  2.5   1.20  0.76     0.9565  (high!)\n",
      "    ============== ==== ==== ======= ===== ====================\n",
      "\n",
      "    :Missing Attribute Values: None\n",
      "    :Class Distribution: 33.3% for each of 3 classes.\n",
      "    :Creator: R.A. Fisher\n",
      "    :Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov)\n",
      "    :Date: July, 1988\n",
      "\n",
      "This is a copy of UCI ML iris datasets.\n",
      "http://archive.ics.uci.edu/ml/datasets/Iris\n",
      "\n",
      "The famous Iris database, first used by Sir R.A Fisher\n",
      "\n",
      "This is perhaps the best known database to be found in the\n",
      "pattern recognition literature.  Fisher's paper is a classic in the field and\n",
      "is referenced frequently to this day.  (See Duda & Hart, for example.)  The\n",
      "data set contains 3 classes of 50 instances each, where each class refers to a\n",
      "type of iris plant.  One class is linearly separable from the other 2; the\n",
      "latter are NOT linearly separable from each other.\n",
      "\n",
      "References\n",
      "----------\n",
      "   - Fisher,R.A. \"The use of multiple measurements in taxonomic problems\"\n",
      "     Annual Eugenics, 7, Part II, 179-188 (1936); also in \"Contributions to\n",
      "     Mathematical Statistics\" (John Wiley, NY, 1950).\n",
      "   - Duda,R.O., & Hart,P.E. (1973) Pattern Classification and Scene Analysis.\n",
      "     (Q327.D83) John Wiley & Sons.  ISBN 0-471-22361-1.  See page 218.\n",
      "   - Dasarathy, B.V. (1980) \"Nosing Around the Neighborhood: A New System\n",
      "     Structure and Classification Rule for Recognition in Partially Exposed\n",
      "     Environments\".  IEEE Transactions on Pattern Analysis and Machine\n",
      "     Intelligence, Vol. PAMI-2, No. 1, 67-71.\n",
      "   - Gates, G.W. (1972) \"The Reduced Nearest Neighbor Rule\".  IEEE Transactions\n",
      "     on Information Theory, May 1972, 431-433.\n",
      "   - See also: 1988 MLC Proceedings, 54-64.  Cheeseman et al\"s AUTOCLASS II\n",
      "     conceptual clustering system finds 3 classes in the data.\n",
      "   - Many, many more ...\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#scikit-learn库内置了该数据集，可直接导入。\n",
    "from sklearn.datasets import load_iris\n",
    "dataset = load_iris()\n",
    "X = dataset.data\n",
    "y = dataset.target\n",
    "print(dataset.DESCR)#查看数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据集的特征为连续值，而我们即将使用的算法使用类别型特征值，因此我们需要把连续值\n",
    "转变为类别型，这个过程叫作离散化。\n",
    "最简单的离散化算法，莫过于确定一个阈值，将低于该阈值的特征值置为0，高于阈值的置\n",
    "为1。我们把某项特征的阈值设定为该特征所有特征值的均值。每个特征的均值计算方法如下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "attribute_means = X.mean(axis=0)\n",
    "#将数据集打散，把连续的特征值转换为类别型\n",
    "X_d = np.array(X >= attribute_means, dtype='int')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "OneR算法的思路很简单，它根据已有数据中，具有相同特征值的个体最可能属于哪个类别\n",
    "进行分类。OneR是One Rule（一条规则）的简写，表示我们只选取四个特征中分类效果最好的一\n",
    "个用作分类依据。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from collections import defaultdict\n",
    "from operator import itemgetter\n",
    "def train_feature_value(X, y_true, feature, value):\n",
    "    #遍历数据集中每一条数据（代表一个个体），统计具有给定特征值的个体在各个类别中的出现次数。\n",
    "    class_counts = defaultdict(int)\n",
    "    for sample, y in zip(X, y_true):\n",
    "        if sample[feature] == value:\n",
    "            class_counts[y] += 1\n",
    "    # 对class_counts字典进行排序，找到最大值，就能找出具有给定特征值的个体在哪个类别中出现次数最多。\n",
    "    sorted_class_counts = sorted(class_counts.items(),key=itemgetter(1), reverse=True)\n",
    "    most_frequent_class = sorted_class_counts[0][0]\n",
    "    n_samples = X.shape[1]\n",
    "    #计算该条规则的错误率\n",
    "    error = sum([class_count for class_value, class_count in class_counts.items() if class_value != most_frequent_class])\n",
    "    #返回使用给定特征值得到的待预测个体的类别和错误率\n",
    "    return most_frequent_class, error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "定义一个函数，对于某项特征，遍历其每一个特征值，使用上述函数，就能得到预测结果和每个特征值所带\n",
    "来的错误率，然后把所有错误率累加起来，就能得到该特征的总错误率。\n",
    "函数声明如下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def train_on_feature(X, y_true, feature_index):\n",
    "    # 找出给定特征共有几种不同的取值。\n",
    "    values = set(X[:,feature_index])\n",
    "    # 再创建字典predictors，用作预测器。字典的键为特征值，值为类别。比如键为1.5、值为2，表示特征值为1.5的个体属于类别2。\n",
    "    # 创建errors列表，存储每个特征值的错误率。\n",
    "    predictors = {}\n",
    "    errors = []\n",
    "    for current_value in values:\n",
    "        most_frequent_class, error = train_feature_value(X,y_true, feature_index, current_value)\n",
    "        predictors[current_value] = most_frequent_class\n",
    "        errors.append(error)\n",
    "    #最后，计算该规则的总错误率，返回预测器及总错误率\n",
    "    total_error = sum(errors)\n",
    "    return predictors, total_error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "机器学习流程分为两步：训练和测试。在训练阶段，我们从数据集中取一部分数\n",
    "据，创建模型。在测试阶段，我们测试模型在数据集上的分类效果。考虑到模型的目标是对新个\n",
    "体进行分类，因此不能用测试数据训练模型，因为这样做容易导致过拟合问题。\n",
    "过拟合指的是模型在训练集上表现很好，但对于没有见过的数据表现很差。解决方法很简单：\n",
    "千万不要用训练数据测试算法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The best model is based on variable 2 and has error 37.00\n",
      "{'variable': 2, 'predictor': 0}\n"
     ]
    }
   ],
   "source": [
    "# scikit-learn库提供了一个将数据集切分为训练集和测试集的函数。\n",
    "from sklearn.cross_validation import train_test_split\n",
    "#该函数根据设定的比例（默认把数据集的25%作为测试集）将数据集随机切分为两部分，以确保测试结果的可信度。\n",
    "Xd_train, Xd_test, y_train, y_test = train_test_split(X_d, y,random_state=14) # random_state 为随机种子 .默认不需要设置\n",
    "#得到两个数据集：训练集Xd_train和测试集Xd_test。y_train和y_test分别为以上两个数据集的类别信息。\n",
    "#计算所有特征值的目标类别（预测器）。记得只使用训练集。遍历数据集中的每个特征，使用我们先前定义的函数train_on_feature()训练预测器，计算错误率。\n",
    "all_predictors = {}\n",
    "errors = {}\n",
    "for feature_index in range(Xd_train.shape[1]):\n",
    "    predictors, total_error = train_on_feature(Xd_train, y_train,feature_index)\n",
    "    all_predictors[feature_index] = predictors\n",
    "    errors[feature_index] = total_error\n",
    "# 然后找出错误率最低的特征，作为分类的唯一规则。\n",
    "best_variable, best_error = sorted(errors.items(), key=itemgetter(1))[0]\n",
    "print(\"The best model is based on variable {0} and has error {1:.2f}\".format(best_variable, best_error))\n",
    "# 对预测器进行排序，找到最佳特征值，创建model模型。\n",
    "model = {'variable': best_variable,'predictor': all_predictors[best_variable][0]}\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 通过遍历数据集中的每条数据来完成预测\n",
    "def predict(Xd_test, model):\n",
    "    variable = model['variable']\n",
    "    predictor = model['predictor']\n",
    "    y_predicted = []\n",
    "    for sample in Xd_test:\n",
    "        y_predicted.append(int(sample[variable]))\n",
    "    return y_predicted"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The test accuracy is 76.3%\n"
     ]
    }
   ],
   "source": [
    "# 用上面这个函数预测测试集中每条数据的类别。\n",
    "y_predicted = predict(Xd_test, model)\n",
    "# 比较预测结果和实际类别，就能得到正确率是多少。\n",
    "accuracy = np.mean(y_predicted == y_test) * 100\n",
    "print(\"The test accuracy is {:.1f}%\".format(accuracy))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
